Papers with few-shot adaptation
Systematic Analysis for Pretrained Language Model Priming for Parameter-Efficient Fine-tuning (2024.naacl-srw)
Copied to clipboard
| Challenge: | Parameter-efficient (PE) methods for adapting pre-trained language models to downstream tasks are still lacking in many cases. |
| Approach: | They propose a general PE priming framework to enhance few-shot adaptation and generalization ability of PE methods. |
| Outcome: | The proposed framework reveals that the best priming strategy facilitates adaptation to target tasks. |
Prompts Can Play Lottery Tickets Well: Achieving Lifelong Information Extraction via Lottery Prompt Tuning (2023.acl-long)
Copied to clipboard
| Challenge: | Existing research on information extraction tasks focuses on one specific task, but in real-world scenarios, new data of different IE tasks and domains come in a stream over time. |
| Approach: | They propose a parameter- and deployment-efficient prompt tuning method to evaluate the UIE system under a “lifelong learning” setting. |
| Outcome: | The proposed method is able to learn new tasks without forgetting old ones and expand knowledge and functionalities without retraining the whole system. |
On Measuring the Intrinsic Few-Shot Hardness of Datasets (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent work has shown that few-shot learning is successful for pre-trained models, but there is no concrete understanding of when and why it is successful. |
| Approach: | They propose a simple metric that estimates few-shot hardness for a given dataset . they propose metric which exploits feature-space invariances between training and test samples . |
| Outcome: | The proposed metric better accounts for few-shot hardness compared to existing notions and is 8-100x faster to compute. |
How to Solve Few-Shot Abusive Content Detection Using the Data We Actually Have (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing datasets for abusive language detection are expensive and lack of knowledge about the target is a challenge. |
| Approach: | They propose to build models cheaply for a new target label set and/or language, using only a few training examples of the target domain. |
| Outcome: | The proposed model improves monolingually and across languages using existing datasets and only a few-shots of the target domain. |
Impressions: Visual Semiotics and Aesthetic Impact Understanding (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing image captioning and conditional generation models struggle to simulate plausible human responses to images. |
| Approach: | They propose a dataset to investigate the semiotics of images and how visual features and design choices can elicit specific emotions, thoughts and beliefs. |
| Outcome: | The proposed dataset improves existing models for image captioning and conditional generation. |
HyperLoRA: Efficient Cross-task Generalization via Constrained Low-Rank Adapters Generation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to adapt pre-trained language models (PLMs) to emerging tasks are costly and inefficient. |
| Approach: | They propose a meta-network that generates task-specific weights without any optimization. |
| Outcome: | The proposed approach has flexible generalization ability and superior performance over hypenetworks. |